
PleaseTypB-fftJTus sign (+) inside this box 7 | + | 



ER622185?29 



r 



PTO/SB/21 (08-00) 
Approved for use tlirough 10/31/2002. 0MB 0651-0031 
U.S. Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE 
Under the Papenwork Reduction Act of 1995. no persons are required to respond to a collection of information unless it displays a valid 0MB control number. 



TRANSMITTAL 
FORM 

(to be used for all correspondence after initial filing) 



Application Number 


\ 

09/805,626 


Filing Date 




First Named Inventor 


Zhang ( 


Group Art Unit 


^ - FEB 3 2004 

2171 ; 


Examiner Name 


THAI, HAN J§chnology Center 


Attorney Docket Number 


MS1-725US y 



noo 



Total Number of Pages in This Submission 



ENCLOSURES (check all that apply) 



0 

□ 



□ 
□ 

□ 
□ 



Fee Transmittal Form 
I I Fee Attached 

Amendment / Reply 

I I After Final 

I I Affidavits/declaration(s) 

Extension of Time Request 

Express Abandonment Request 

Information Disclosure Statement 

Certified Copy of Priority 
Document(s) 

Response to Missing Parts/ 
Incomplete Application 

□ Response to Missing Parts 
under 37 CFR1.52 or 1.53 



□ 
□ 
□ 
□ 
□ 

□ 



□ 

□ 



Assignment Papers 
(for an Application) 

Drawing(s) Sheets 
Licensing-related Papers 
Petition 

Petition to Convert to a 
Provisional Application 

Power of Attomey, Revocation 
Change of Correspondence 
Address 

Terminal Disclaimer 
Request for Refund 

CD, Number of CD(s) 



Remarks 



□ After Allowance Communication 
to Group 

0 Appeal Communication to Board 
of Appeals and Interferences 

□ Appeal Communication to Group 
{Appeal Notice. Brief, Reply Brief) 



□ 
□ 
0 



Proprietary Information 
Status Letter 

Other EncIosure{s) (please 
identify below): 

Appeal in Triplicate w/Appendix 
of Claims on Appeal; and 
Return Receipt Postcard; 



SIGNATURE OF APPLICANT, ATTORNEY, OR AGENT 



Fimi 
or 

Individual name 



David 8. Lee 



Signature 




Date 



fel^u^r^ fZ ^^^^ 



CERTIFICATE OF EXPRESS MAILING 



I hereby certify that the Items listed above as enclosed are being deposited with the U.S. Postal Sen/Ice as Express Mall, in an envelope addressed to 

The Commissioner of Patents , Alexandria, VA 22313 on the below-Indicated date. The Express Mall No. has also been marked on the listed Items. 

Express Mail No. ER 622185325US 



Typed or printed name 



Wendy A. Saxby 



1^ 



Signature 



Date 



Burden Hour Statement: This form is estimated to take O(^hours to complete/^ime will vary depending upon the needs of the individual case. Any comments 
on the amount of time you are required to complete this form should be serfto the Chief Information Officer, U.S. Patent and Trademark Office, Washington, 
DC 20231. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. 



Under thg 




ER622185329' 



PTO/SB/17 (10-03) 
Approved for use through 07/31/2006. 0MB 0651-0032 
U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 
[ctj3fJ995. no persons are required to resp ond to a collection of informati on unless it displays a valid 0M B control number . 



ANSMITTAL 



for FY 2004 

Effective 1010112003, Patent fees are subject to annual revision. 



I I Applicant claims small entity status. See 37 CFR 1 .27 



TOTAL AMOUNT OF PAYMENT 



($) 330.00 



Complete if Known 



Application Number 



Filing Date 



First Named Inventor 



Examiner Name 



Art Unit 



Attorney Docket No. 



09/805.626 



Mar 13. 2001 



Zhang 



RECEIVC P 



THAI, HANH B 



2171 



FEB ? 3 2001 



MS1-725US 



Tachnology CROlef 



2100 



METHOD OF PAYMENT (check all that apply) 



FEE CALCULATION (continued) 



rn Check Q Credit card Monkey other None 

0 Deposit Account: 



3. ADDITIONAL FEES 



Large Entity 



Deposit 
Account 
Number 
Deposit 
Account 
Name 



12-0769 



Lee & Hayes, PLLC 



The Director is authorized to: (check all that apply) 

0 Charge fee(s) indicated below [✓! Credit any overpayments 

[3 Charge any additional fee(s) or any underpayment of fee(s) 

Q Charge fee(s) indicated below, except for the fiiing fee 

to the above-identified deposit account. 



Fee Fee 
Code ($) 

1051 130 

1052 50 

1053 130 
1812 2,520 

1804 920^ 

1805 1,840* 



FEE CALCULATION 



I.BASIC FILING FEE 

Large Entity Small Entity 



1251 
1252 
1253 



110 
420 
950 



Fee Fee 

bode ($) 

1001 770 

1002 340 

1003 530 

1004 770 

1005 160 



Fee Fee 

^ode ($) 

2001 385 

2002 170 

2003 265 

2004 385 

2005 80 



Fee Description 

Utility filing fee 
Design filing fee 
Plant filing fee 
Reissue filing fee 
Provisional filing fee 

SUBTOTAL (1) 



Fee Paid 



($)0 



1254 1,480 

1255 2,010 



1401 
1402 
1403- 



330 
330 
290 



2. EXTRA CLAIM FEES FOR UTILITY AND REISSUE 

Fee from 

Ext ra Claim s below Fee Paid 



1451 1,510 

1452 110 

1453 1,330 
1501 1,330 



Total Claims |_ 



Independent I 
Claims I — 
Multiple Dependent 



] -20* 
1 -3** 



xL 



Large Entity 


Small Entity 


Fee 


Fee 


Fee Fee 


Code 


($) 


Code ($) 


1202 


18 


2202 9 


1201 


86 


2201 43 


1203 


290 


2203 145 


1204 


86 


2204 43 


1205 


18 


2205 9 



Fee Description 

Claims in excess of 20 

independent claims in excess of 3 

Multiple dependent claim, if not paid 

** Reissue independent claims 
over original patent 

** Reissue claims in excess of 20 
and over original patent 



1502 
1503 
1460 
1807 
1806 
8021 
1809 



480 
640 
130 

50 
180 

40 
770 



1810 770 



1801 
1802 



770 
900 



Small Entity 



Fee Description 



Fee Fee 
Code ($) 

2051 65 Surcharge - late filing fee or oath 

2052 25 Surcharge - late provisional filing fee or 

cover sheet 
1053 130 Non-English specification 
1812 2,520 For filing a request for ex parte reexamination 

1804 920* Requesting publication of SIR prior to 

Examiner action 

1805 1,840* Requesting publication of SIR after 

Examiner action 

2251 55 Extension for reply within first month 

2252 210 Extension for reply within second month 

2253 475 Extension for reply within third month 

2254 740 Extension for reply within fourth month 

2255 1,005 Extension for reply within fifth month 
2401 165 Notice of Appeal 

1 65 Filing a brief in support of an appeal 
145 Request for oral hearing 
1,510 Petition to institute a public use proceeding 
55 Petition to revive - unavoidable 
665 Petition to revive - unintentional 
665 Utility issue fee (or reissue) 
240 Design issue fee 
320 Plant issue fee 
130 Petitions to the Commissioner 
50 Processing fee under 37 CFR 1.1 7(q) 
180 Submission of Information Disclosure Stmt 



2402 
2403 
1451 
2452 
2453 
2501 
2502 
2503 
1460 
1807 
1806 
8021 
2809 

2810 

2801 
1802 



Recording each patent assignment per 
property (times number of properties) 

385 Filing a submission after final rejection 
(37 CFR 1.129(a)) 

385 For each additional invention to be 
examined (37 CFR 1.129(b)) 

385 Request for Continued Examination (RCE) 

900 Request for expedited examination 
of a design application 



'*or number previously paid, if greater; 



SUBTOTAL (2) 

// greater: For Reissues, see above 



ml 



other fee (specify) 

*Reduced by Basic Filing Fee Paid 



330.00 



SUBTOTAL (3) 



330.00 



SUBiyilTTED BY 



[Registration No. 
fAttomev/Aaent) 



(Complete (if applicable)) 



Name (Print/Type) 



David S. Lee 



38,222 



Telephone (206) 315-4001 



^ignatur^ 



WARNING: Information on this form may become public. Credit card information should not 
be included on this form. Provide credit card information and authorization on PTO-2038. 

This collection of information is required by 37 CFR 1.17 and 1.27. The information is required to obtain or retain a benefit by the public which is to file (and by the 
USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, 
including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on 
the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and 
Trademark Office. U.S. Department of Commerce, P.O. Box 1450. Alexandria. VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. 
SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450. 

If you need assistance in completing the form, call 1'dOO-PTO-9199 and select option 2. 




\ ^ 2JJ04 

i eR622l85325 

^ THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Application Serial No.: 09/805,626 

Filing Date: March 13, 2001 

Inventors: Zhang et al. 

Appellant: Microsoft Corporation 

Group Art Unit: 2171 

Examiner: THAI, H.B. 

Confirmation No. : 3014 

Attorney's Docket No.: MS1-725US 

Title: A Media Content Search Engine Incorporating Text Content 

and User Log Mining 

APPEAL BRIEF RECEIVED 

FEB ?, 3 2004 

To: Mail Stop Appeal Brief - Patent 

Commissioner for Patents Technology Center 21 00 

P.O. Box 1450 
Alexandria, VA 22313-1450 



From: David S. Lee 

Customer No.: 22801 

Sir: 

In response to the Final Office Action of October 14, 2003 in connection 
with the above-identified application, and fiirther to the Notice of Appeal filed on 
January 14, 2004, an Appeal is made. Favorable consideration is respectfully 
requested. 



02/23/2004 mmm OOOOOOSS 120769 09805626 
01 FC:1402 330.00 Dfl 



lee ©ha yes poc &09-324-9256 



1 



REAL PARTY IN INTEREST 

The real party part in interest in the present matter is the Microsoft 
Corporation of Redmond, WA, USA. 

RELATED APPEALS AND INTERFERENCES 

There are no related appeals or interferences known to the Appellant or the 
Appellant's undersigned representative that would directly affect, or be directly 
affected by, the outcome of the present Appeal. 

STATUS OF CLAIMS 

The application that is subject to the present Appeal was filed on March 13, 
2001, and included Claims 1-54. This Appeal is made to the rejection of all of 
Claims 1-54. 

STATUS OF AMENDMENTS 

Subsequent to the Final Office Action of October 14, 2003, the Appellant 
filed a Request for Reconsideration in the U.S. Patent and Trademark Office on 
November 12, 2003. 

An Advisory Action was issued on December 4, 2003, indicating that the 
Request for Reconsideration had been considered but was deemed to not place the 
application in condition for allowance. The Advisory Action indicates that, for the 
purposes of Appeal, Claims 1-54 are rejected. 
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SUMMARY OF THE INVENTION 

The present invention is directed towards searching media content using 
related high-level text features and user log mining. The present summary 
references the example system of Fig. 2 and the example process of Fig. 3 of the 
subject application. 

As set forth in the "Summary of the Invention'^bn page 2 of the application, 
text features corresponding to pieces of media content are extracted from media 
content sources. The media content pieces can be a variety of types of media 
content, such as images, audio, multimedia content, etc. One or more text features 
{e.g,, one or more words) for a piece of media content are extracted from text 
associated with the piece of media content. A text feature vector is generated from 
these extracted text features and made available for comparison to a query vector 
during subsequent searches. Additional low-level feature vectors may also be 
extracted from the piece of media content and used during the comparison process. 

Fig. 2 of the application illustrates an exemplary media content retrieval 
system 120 in accordance with certain embodiments of the invention, and Fig. 3 
shows a flowchart illustrating an exemplary process for collecting and indexing 
pieces of media content from web pages in accordance with certain embodiments of 
the invention. The process of Fig. 3 may be performed in software. 

Referring to the system of Fig. 2, the image retrieval process is initiated by a 
user via a user query interface 180 of user interface 122. Alternatively, the image 
retrieval process may be initiated by another component or module. When a media 
content search request is made, retrieval component 124 receives the search request 
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and accesses component 126 to determine which pieces of media content to return 
in response to the search request. 

Query vectors are generated by interface module 180 or a module of 
retrieval system 124 and stored in query record 182. A low-level query vector may 
be generated by extracting and concatenating the low-level features from the input 
image. A high-level query vector may generated by extracting keywords from the 
search criteria and building a query vector by assigning a value of one to the 
element corresponding to each extracted keyword and a value of zero for the other 
elements. 

A matching module 184 compares the query vectors to the feature vectors in 
a document space model 186 and determines how closely the query vectors match 
the feature vectors in document space model 186. Matching module 184 performs 
its comparison of query vectors to feature vectors based on both the low-level and 
high-level query and feature vectors. Matching module 184 determines which 
images to retum to user interface component 122 as matching or satisfying the 
search criteria in a variety of different manners. The user is given the opportunity, 
via user feedback interface 188, to indicate, for each returned image, whether the 
image is relevant or irrelevant. Once input, this relevance feedback is stored in a 
user log 190. 

Query updating module 192 accesses the relevance feedback from user log 
190 and updates the query vectors in query vector 182 to reflect the relevance 
feedback provided by the user. Once the new query vectors are generated, 
matching module 184 repeats its comparison process using the updated query 
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vectors, and returns a new set of closely matching media content pieces to user 
interface component 122. This feedback process can be repeated multiple times. 

ISSUES 

The issues presented for appeal include: 

1) the rejection of Claim 40 under 35 U.S.C. § 102(e) as being anticipated 
by Marchisio (U.S. Patent No. 6,510,406; hereafter "Marchisio"); 

2) the rejection of Claims 1-39 and 41-50 under 35 U.S.C. §103(a) as being 
unpatentable over Marchisio in view of Hoffert et al. (U.S. Patent 6,282,549; 
hereafter "Hoffert"); 

3) the rejection of Claims 51-53 under over Marchisio in view of Ma et al. 
(U.S. Patent 6,347,313; hereafter "Ma"); and 

4) the rejection of Claim 54 under 35 U.S.C. §103(a) as being unpatentable 
over Marchisio in view of Ma in view of Hoffert. 

GROUPING OF CLAIMS 

The Appellant submits that the claims under appeal do not stand or fall 
together. The rejections presented in the Final Office Action are addressed below 
as they pertain to the following groups of claims. 

a. Claim 40; 

b. Claim 1,5-9; 

c. Claim 2; 

d. Claims 3 and 4; 

e. Claims 10-26; 
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f. Claims 27, 28, and 32-39; 

g. Claim 29; 

h. Claims 30 and 31; 

i. Claim 41; 

j. Claims 42-47; 

k. Claims 48-50; 

1. Claims 51-53; and 

m. Claim 54. 

ARGUMENT 

1) The rejection of Claim 40 under 35 U.S.C. § 102(e) 

a. The Appellant, respectfully submits that Marchisio does not 
anticipate Claim 40. 

Marchisio relates to a latent semantic-based information retrieval system 
(Marchisio, col. 4, lines 66 and 67). As described in the Summary, Marchisio 
describes document processing steps to pre-process searchable documents to 
generate a representation of a search space, and further performs query 
processing steps to process a search query received from a user to generate a 
query vector for the query (Marchisio, col. 5, lines 8-14). The system includes 
an indexing module, storage module, search module, and query module 
(Marchisio, Fig. 2 and col. 8, lines 38-33). 

However, Marchisio does not anticipate Claim 40 under the requirements 
set forth in MPEP §2131, which clearly states that the reference must teach every 
element of the claim and also be arranged as required by the rejected claim. 
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Tellingly, these essential requirements are found in the opening discussion of the 
standards for an anticipation rejection in MPEP §2131: 

"The identical invention must be shown in as 
complete detail as is contained in the... claim." 
Richardson v. Suzuki Motor Co,, 868 F.2d 1226, 
1236, 9 USPQ2d 1913, 1920 (Fed. Cir. 1989). The 
elements must be arranged as required by the claim, 
but this is not an ipsissimis verbis test, i.e., identity of 
terminology is not required. In re Bond, 910 F.2d 
831, 15 USPQ2d 1566 (Fed. Cir. 1990). 

However, the elements of Marchisio cited in the rejection as anticipating 
those of Claim 40 are not "shown in as complete detail as is contained" in Claim 
40, as required by MPEP §2131. For instance, the rejection contemplates search 
module 24 (Marchisio, Fig. 2) as corresponding to the claimed crawler module. 
But Marchisio does not describe search module 24 as being "coupled to access a 
media content source and collect a plurality of media content pieces and 
associated text from the media content source," as recited in Claim 40. 

Rather, Fig. 2 of Marchisio shows search module 24 being connected to 
storage module 22, which includes a relational database management system 
(RDBMS) for storing a term-document index (col. 9, lines 7-11). A term- 
document matrix is defined as an output of a feature extraction module including 
a number of inverted files (col. 9, lines 32-36). More specifically, the content of 
storage module 22 includes the content of documents 27 that has already been 
reduced by feature extraction module 21 and concept synchronizer 28 into a 
term-document matrix or query vector. Thus, not only does the coupling of 
search module 24 and storage module 22 not anticipate the claimed crawler 
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module, but the arrangement of the search module 24 and storage module 22 
actually teach a processing flow that is opposite to that of Claim 40. 

Thus, not only are the features of Marchisio not "shown in as complete 
detail as is contained" in Claim 40, but neither are the features of the reference 
"arranged as required" by Claim 40, as standardized by MPEP §2131. More 
particularly, as indicated above, Claim 40 recites a system comprising: 

a crawler module coupled to access a media 
content source and collect a plurality of media 
content pieces and associated text from the media 
content source; 

a feature extraction module coupled to extract 
one or more text features from one of the media 
content pieces; and 

a media content indexing module coupled to 
generate a text feature vector, based on the extracted 
one or more text features, corresponding to the one 
media content piece. 

On the other hand, as set forth above, search module 24 is connected to storage 
module 22, which includes an RDBMS for storing a term-document index. 
Thus, search module 24 and search module 21 described by Marchisio are not 
arranged as the crawler module and feature extraction module as recited in Claim 
40. 

Further, the rejection in the Final Office Action attempts to compare the 
feature extraction module of Claim 40 with feature extraction modules 21 of 
Marchisio. However, in Fig. 2 of Marchisio, feature extraction modules 21 in 
indexing module 20 input original documents and a query to concept 
synchronizer 28 (col. 8, lines 64-66), which clearly does not anticipate extracting 
one or more text features from a media content piece as recited in connection 
with the feature extraction module of Claim 40. Thus, feature extraction 
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modules 21 of indexing module 20 do not anticipate the feature extraction 
module recited in Claim 40. 

Further, Fig. 2 of Marchisio also shows a feature extraction module 21 
corresponding to search module 24. This embodiment of feature extraction 
module 21 transmits queries from clients 25 utilizing CORBA (Common Object 
Request Broker Architecture), which helps transfer messages to and from objects 
between various platforms in a distributed environment. Such depiction of 
feature extraction module 21 in search module 24 also fails to anticipate the 
feature extraction module recited in Claim 40. 

Lastly, the rejection attempts to compare the indexing module 20 
described by Marchisio with the media content indexing module recited in Claim 
40. Indexing module 20 in Fig. 2 of Marchisio reduces original documents and a 
query received from the aforementioned extraction modules 21 into symbolic 
form, /.^., a term-document matrix or query vector. Such description does not 
teach a text feature vector being generated based on an extracted text feature 
corresponding to a collected media content piece, as recited in Claim 40. 

Therefore, for at least the reasons set forth above, it is respectfully 
submitted that the rejection of Claim 40 under 35 U.S.C. § 102(e) should be 
reversed. 

2) The rejection of Claims 1-39 and 41-50 under 35 U.S.C. SlOSfa) 
The Appellant respectfully submits that Claims 1-39 and 41-50 are not 
rendered obvious by the combination of Marchisio and Hoffert. 



leeOhayes poc »9-3Z4-92S6 



9 



MPEP §2143 states, in part: 



To establish a prima facie case of obviousness, three 
basic criteria must be met. First, there must be some 
suggestion or motivation, either in the references 
themselves or in the knowledge generally available to 
one of ordinary skill in the art, to modify the 
reference or to combine reference teachings. Second, 
there must be a reasonable expectation of success. 
Finally, the prior art reference (or references when 
combined) must teach or suggest all the claim 
limitations. The teaching or suggestion to make the 
claimed combination and the reasonable expectation 
of success must both be found in the prior art, and not 
based on applicant's disclosure. In re Vaeck, 947 
F.2d 488, 20 USPQ2d 1438 (Fed. Cir. 1991). 

It is respectfully submitted that the requirements for establishing a prima facie 
case of obviousness, set forth in MPEP §2143, have not all been met in the 
present rejection. 



b. With regard to rejected Claim 1, the Appellant submits that the 
proposed combination is not sufficiently motivated by either of Marchisio or 
Hoffert, nor does the proposed combination of Marchisio and Hoffert teach or 
suggest all of the claim limitations. 

On page 6 of the Final Office Action, it is acknowledged that Marchisio 
does not teach "the text feature vectors associated with the plurality of media 
content pieces." Accordingly, Marchisio fails to teach "identifying media 
content pieces to be rendered by comparing the query vector to text features 
associated with the plurality of media content pieces," as recited in Claim 1, and 
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consequently Marchisio provides no basis for the further recited "receiving user 
feedback regarding the relevancy of the identified media content pieces"; 

On the other hand, Hoffert does describe, "that relevant textual 
information may be directly surrounding the media reference on a web page, or it 
may be far from the media reference," (col. 4, lines 55-57). How^ever, neither 
Marchisio nor Hoffert provides any description that suggests that such "relevant 
textual information" may be used in combination with a query vector, as 
described by Marchisio, to identify media content pieces to be rendered, as in 
Claim 1. Further, Hoffert does not compensate for consequences of the 
acknowledged deficiencies of Marchisio, which are noted above. 

Claim 1 compares a query vector to the text feature vector associated with 
a plurality of media content pieces. Arguendo, even if the textual information 
described by Hoffert does suggest the claimed "text feature vectors associated 
with the plurality of media content pieces," Marchisio still would not utilize the 
textual information in such a manner to render the claim obvious. Instead, 
Marchisio parses an electronic text received in a user query (Marchisio, Fig. 1, 
step 16) by recognizing acronyms and extracting word roots to generate a user 
query vector (Marchisio, Fig. 1, step 17, emphasis added here). 

Further still, Marchisio and Hoffert are silent with regard to user feedback 
on the relevancy of identified media content pieces, and the rejection does not 
advance any arguments to the effect that Hoffert is able to remedy such 
deficiency. 

Specifically, Marchisio does not teach or suggest, "receiving user 
feedback regarding the relevancy of the identified media content pieces," as 
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recited in Claim 1, and therefore provides no basis for the further recitation of 
"modifying the query vector based on the user feedback." 

Marchisio does, however, describe "semantic keyword feedback" obtained 
by isolating positive and negative coefficients in a truncated basis function 
expansion for a query approximation (col. 15, lines 57-59), and then describes (at 
col. 19, lines 4-7) an inverse inference engine providing concept feedback 
specific to each partition of a term-document matrix is defined as an output of a 
feature extraction module including a number of inverted files (col. 9, lines 32- 
36). But the feedback described by Marchisio is not utilized for "modifying the 
query vector based on the user feedback," or for modifying whatever is alleged to 
be substituted for text feature vectors associated with the plurality of media 
content pieces, as claimed. 

Therefore, for at least the reasons set forth above, it is respectfully 
submitted that Claim 1 is distinguishable over the proposed combination of 
Marchisio and Hoffert. 

Claim 5-9 depend from Claim 1, and are therefore distinguishable over 
Marchisio and Hoffert for at least the reasons set forth above regarding Claim 1 . 
It is noted that the rejection in the Final Office Action does not assert any points 
of rejection that are specific to the features recited in Claim 5. 

€• Claim 2 is distinguishable over Marchisio and Hoffert for the reasons 
set forth above regarding Claim 1, from which Claim 2 depends. Further, the 
assertion in the Final Office Action that Marchisio discloses the features of 
Claim 2 is contradicted by the rejection of Claim 1. That is, in rejecting Claim 1, 
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it is conceded in the Final Office Action that "Marchisio...does not explicitly 
disclose 'the text feature vectors associated with the plurality of media content 
pieces'." Therefore, if the acknowledgement applies to the first recitation of 
such feature in Claim 1, then surely the acknowledgement also applies to the 
second recitation of such feature in dependent Claim 2, which recites, in part, 
"wherein the identifying comprises, comparing the query vector to text feature 
vectors associated with the plurality of media content pieces to generate first 
results." 

In addition, the rejection in the Final Office Action asserts that Marchisio 
describes the "combining, for one of the plurality of media content pieces, the 
first and second results corresponding to the one media content piece" recited in 
Claim 2. However, while the portion of Marchisio cited in the rejection (col. 17, 
lines 47-53, Fig. 10) refers to an initial search and a secondary search, there is no 
teaching or suggestion that the resuhs of such searches are combined. Rather, 
the search query for the secondary search request combines the selected related 
term or terms and the initial search term (Marchisio, col. 17, lines 37-39). There 
is no mention whatsoever that the user is presented with the results of both 
searches, much less that the results are combined in any manner. 

The Appellant respectfully submits that Hoffert does not compensate for 
such deficiency, nor is any argument offered to that effect. 

d. Claim 3 depends from Claim 2, and is therefore distinguishable over 
Marchisio and Hoffert for the reasons set forth above regarding both Claims 1 
and 2. Further, Marchisio does not alter weights of the results of the first and 
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second searches, as asserted in the rejection in the Final Office Action. Rather, 
Marchisio describes electronic information files from the first search being 
presented to the user (Marchisio, col. 17, lines 54-61) and, in response to a user 
selection or indication, at least one electronic information file associated with the 
document weights generated in response to the secondary searches being 
retrieved (Marchisio, col. 17, lines 61-66). These descriptions do not suggest the 
"combining" of Claim 2, and therefore are unable to "alter" any weights used 
therein. Hoffert does not compensate for such deficiencies of Marchisio, with 
regard to Claim 3. 

Claim 4 depends from Claim 3, and is therefore distinguishable over 
Marchisio and Hoffert for at least the reasons set forth above regarding Claims 1- 
3. 

e. With regard to the method of Claim 10, on page 8 of the Final Office 
Action it is acknowledged that Marchisio does not teach "the text feature vectors 
associated with the plurality of media content pieces." However, such feature is 
not recited in Claim 10. Assuming arguendo that such acknowledgement 
actually refers to the recitation of "extracting, for a piece of media content, one 
or more text features from the associated text," which is found in Claim 10, it is 
submitted that Marchisio provides no basis for the further recitation of "making 
the one or more text features available for searching." 

Hoffert does not compensate for the deficiency acknowledged with regard 
to Marchisio. Hoffert describes, "that relevant textual information may be 
directly surrounding the media reference on a web page, or it may be far from the 
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media reference," (col. 4, lines 55-57). However, Marchisio parses an electronic 
text received in a user query (Marchisio, Fig. 1, step 16, emphasis added here) by 
recognizing acronyms and extracting word roots to generate a user query vector 
(Marchisio, Fig. 1, step 17). Therefore, there is no place for the relevant textual 
information described by Hoffert in the user query processing described by 
Marchisio (Marchisio, Fig. 1). 

Claims 11-26 depend from Claim 10, and are therefore distinguishable 
over Marchisio and Hoffert for at least the reasons set forth above regarding 
Claim 10. 

f. With regard to the method of Claim 27, on page 12 of the Final Office 
Action it is acknowledged that Marchisio does not teach "the text feature vectors 
associated with the plurality of media content pieces." However, such feature is 
not recited in Claim 27. Assuming arguendo that such acknowledgement is 
made with regard to the recitation of "comparing the query vector to a feature 
vector corresponding to a piece of media content and having been generated 
based on text associated with the piece of media content," which is found in 
Claim 27, it is submitted that Marchisio provides no basis for the further 
recitation of "determining, based at least in part on a result of the comparing, 
whether to render the piece of media content to a user." 

Hoffert does not compensate for the deficiency acknowledged with regard 
to Marchisio. Hoffert describes, "that relevant textual information may be 
directly surrounding the media reference on a web page, or it may be far from the 
media reference," (col. 4, lines 55-57). However, Marchisio parses an electronic 
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text received in a user query (Marchisio, Fig. 1, step 16, emphasis added here) by 
recognizing acronyms and extracting word roots to generate a user query vector 
(Marchisio, Fig. 1, step 17). Therefore, there is no place for the relevant textual 
information described by Hoffert in the user query processing described by 
Marchisio (Marchisio, Fig. 1). 

Claims 28 and 32-39 depend from Claim 27, either directly or indirectly, 
and are therefore distinguishable over Marchisio and Hoffert for at least the 
reasons set forth above regarding Claim 27. It is noted that the rejection in the 
Final Office Action includes no specific points of rejection with regard to Claims 
28 and 32. 

g. Claim 29 depends from Claim 27, and is therefore distinguishable 
over Marchisio and Hoffert for at least the reasons set forth above regarding 
Claim 27. Further, the acknowledgement on page 12 of the Final Office Action 
that Marchisio does not explicitly disclose "the text feature vectors associated 
with the plurality of media content pieces," with regard to Claim 27, is also 
applicable to the second recitation of such feature in dependent Claim 29, which 
recites, in part, "comparing the other query vector to another feature vector 
corresponding to the piece of media content and having been generated based on 
the one or more low-level features of the piece of media content." Thus, 
Marchisio and Hoffert fail to render Claim 29 obvious. 

h. Claim 30 depends from Claim 29, and is therefore distinguishable 
over Marchisio and Hoffert for at least the reasons set forth above regarding 
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Claims 27 and 29. Further, Marchisio does not alter weights of the resuhs based 
on user feedback, as asserted in the rejection in the Final Office Action. Rather, 
Marchisio describes electronic information files from the first search being 
presented to the user (Marchisio, col. 17, lines 54-61) and, in response to a user 
selection or indication, at least one electronic information file associated with the 
document weights generated in response to the secondary searches being 
retrieved (Marchisio, col. 17, lines 61-66). These descriptions do not suggest the 
"combining" of Claim 29, and therefore are unable to "alter" any weights used 
therein. 

Claim 31 depends from Claim 30, and is therefore distinguishable over 
Marchisio and Hoffert for at least the reasons set forth above regarding Claims 
27, 29, and 30. 

i. With regard to rejected Claim 41, the Appellant submits that the 
proposed combination is not sufficiently motivated by either of Marchisio or 
Hoffert, nor does the proposed combination of Marchisio and Hoffert teach or 
suggest all of the claim limitations. 

In the Final Office Action, it is acknowledged that Marchisio does not 
teach "the text feature vectors associated with the plurality of media content 
pieces." Accordingly, Marchisio fails to teach "receive the query vector and 
compare the query vector to a plurality of feature vectors corresponding to a 
plurality of pieces of media content, wherein each of the plurality of feature 
vectors has been generated based on text associated with one of the plurality of 
pieces of media content," as recited in Claim 41. Consequently, Marchisio 
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provides no basis for "identify one or more of the plurality of pieces of media 
content to return for rendering to a user based on the comparison of the query 
vector to the plurality of feature vectors/' which is recited further in Claim 41. 

Claim 41 compares a query vector to the text feature vector associated 
with a plurality of media content pieces. Arguendo, even if the textual 
information described by Hoffert does suggest the claimed "text feature vectors 
associated with the plurality of media content pieces," Marchisio still would not 
utilize the textual information in such a manner to render the claim obvious. 
Instead, Marchisio parses an electronic text received in a user query (Marchisio, 
Fig. 1, step 16) by recognizing acronyms and extracting word roots to generate a 
user query vector (Marchisio, Fig. 1, step 17, emphasis added here). 

j. With regard to the method of Claim 42, on page 15 of the Final Office 
Action it is acknowledged that Marchisio does not teach "the text feature vectors 
associated with the plurality of media content pieces." However, such feature is 
not recited in Claim 42. Assuming arguendo that such acknowledgement was 
made with regard to the recitation of "weighting for another piece of media 
content, based on the user feedback, both a result of comparing the high-level 
query vector to a high-level feature vector of the other piece of media content 
and a result of comparing the low-level query vector to a low-level feature vector 
of the other piece of media content," which is found in Claim 42, it is submitted 
that Marchisio provides no basis for the further recitation of "combining the 
weighted result to determine whether to identify the other piece of media content 
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for rendering." Hoffert does not compensate for the deficiency acknowledged 
with regard to Marchisio. 

Further still, neither Marchisio nor Hoffert weights the results of other 
searches, as asserted in the rejection in the Final Office Action. Rather, 
Marchisio describes electronic information files from the first search being 
presented to the user (Marchisio, col. 17, lines 54-61) and, in response to a user 
selection or indication, at least one electronic information file associated with the 
document weights generated in response to the secondary searches being 
retrieved (Marchisio, col. 17, lines 61-66). These descriptions do not suggest the 
"combining" of Claim 42. 

Claims 43-47 depends from Claim 42, and are therefore distinguishable 
over Marchisio and Hoffert for at least the reasons set forth above regarding 
Claim 42, 

k. With regard to rejected Claim 48, the Appellant submits that the 
proposed combination is not sufficiently motivated by either of Marchisio or 
Hoffert, nor does the proposed combination of Marchisio and Hoffert teach or 
suggest all of the claim limitations. 

As set forth above regarding Claims 1 and 41, on page 6 of the Final 
Office Action, it is acknowledged that Marchisio does not teach "the text feature 
vectors associated with the plurality of media content pieces." Accordingly, 
Marchisio fails to teach "modifying, based on the user feedback, a feature vector 
corresponding to the piece of media content," as recited in Claim 48. 
Consequently, Marchisio provides no basis for the further recited "making the 
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modified feature vector available for subsequent searching of the one or more 
pieces of media content". 

Hoffert describes, "that relevant textual information may be directly 
surrounding the media reference on a web page, or it may be far from the media 
reference," (col. 4, lines 55-57). However, neither Marchisio nor Hoffert has any 
description to suggest that such "relevant textual information" may be used for 
subsequent searching of pieces of media content, as in Claim 48. 

Further still, Marchisio and Hoffert are silent with regard to user feedback 
on the relevancy of identified media content pieces, and the rejection does not 
advance any arguments to the effect that Hoffert is able to remedy such 
deficiency. Specifically, Claim 48 recites "modifying, based on the user 
feedback, a feature vector corresponding to the piece of media content." 
Marchisio does not teach or suggest, "receiving user feedback regarding the 
relevancy of one or more pieces of rendered media content," as further recited in 
Claim 48. 

Marchisio does describe "semantic keyword feedback" obtained by 
isolating positive and negative coefficients in a truncated basis function 
expansion for a query approximation (col. 15, lines 57-59), and then describes (at 
col. 19, lines 4-7) an inverse inference engine providing concept feedback 
specific to each partition of a term-document matrix is defined as an output of a 
feature extraction module including a number of inverted files (col. 9, lines 32- 
36). But the feedback described by Marchisio is not utilized for "modifying, 
based on the user feedback, a feature vector corresponding to the piece of media 
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content/' or for modifying whatever is alleged to be substituted for text feature 
vectors associated with the plurality of media content pieces, as claimed. 

Claims 49 and 50 depend from Claim 48, and are therefore 
distinguishable over Marchisio and Hoffert for the reasons set forth above with 
regard to Claim 48. 

It is respectfully submitted that the rejection in the Final Office Action 
attempts to match terms from the rejected claims with those found in the 
proposed references in a piecemeal manner, but fails to provide any teaching that 
would support a combination that teaches the claimed invention. Thus, it is 
requested that the rejection of Claims 1-39 and 41-50 under 35 U.S.C. §103(a) be 
reversed. 

3) The rejection of Claims 51-53 under 35 U.S.C. $103(a) 
The Appellant respectfully submits that Claims 51-53 are not rendered 
obvious by the combination of Marchisio and Ma. 

1. With regard to Claim 51, the Appellant submits that Marchisio parses 
an electronic text received in a user query (Marchisio, Fig. 1, step 16, emphasis 
added here) by recognizing acronyms and extracting word roots to generate a 
user query vector (Marchisio, Fig. 1, step 17). But, there is no teaching in 
Marchisio to even suggest that a feature vector corresponding to a piece of media 
content is used in the user query processing described by Marchisio. That much 
was repeatedly acknowledged in the Final Office Action, by the concession that 
"Marchisio... does not disclose the text feature vectors associated with the 
plurality of media content pieces," with regard to the rejection of Claims 1-39 
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and 41-50 under 35 U.S.C. §103(a). It is further submitted that Ma does not 
compensate for such deficiency, nor is an argument to that effect advanced in the 
Final Office Action. 

It is asserted in the rejection in the Final Office Action that Marchisio, col. 
17, lines 55-65 discloses the claimed "modifying the query vector based on the 
received user feedback." However, the cited portion of Marchisio actually 
describes electronic information files from the first search being presented to the 
user (Marchisio, col. 17, lines 54-61) and, in response to a user selection or 
indication, at least one electronic information file associated v^ith document 
weights generated in response to the secondary searches are retrieved (Marchisio, 
col. 17, lines 61-66). The cited description does not teach or suggest a query 
vector being modified in response to user feedback. 

It is further submitted that Ma does not remedy the deficiencies of 
Marchisio in view of Claim 51. In particular, neither Marchisio nor Ma provide 
any teaching as to how the offline user feedback described by Ma (col. 9, lines 
21-41) would be used to modify the query vector or the feature vector, as recited 
in Claim 5 1 . The mere inclusion of user feedback is insufficient to render the 
claim obvious. 

Claim 52 depends from Claim 51, and is therefore distinguishable over 
Marchisio and Ma for at least the reasons set forth above regarding Claim 51. 
Claim 53 depends from Claims 5 1 and 52, and is therefore distinguishable over 
Marchisio and Ma for at least the reasons set forth above regarding Claims 51 
and 52. 
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For at least the reasons set forth above, it is respectfully submitted that the 
rejection of Claims 51-53 under 35 U.S.C. § 103(a) should be reversed. 

4) The rejection of Claim 54 under 35 U.S.C. m3(a) 
The Appellant respectfully submits that Claim 54 is not rendered obvious by 
the combination of Marchisio, Ma, and Hoffert. 

m. Claim 54 depends from Claim 51, and is therefore distinguishable 
over Marchisio and Ma for at least the reasons set forth above regarding Claims 
51 and 52. Further, such deficiencies are not compensated for by Hoffert, nor are 
assertions to that effect made in the Final Office Action. 

Accordingly, for at least the reasons set forth above, it is respectfully 
submitted that the rejection of Claim 54 under 35 U.S.C. § 103(a) should be 
reversed. 
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CONCLUSION 

For at least the reasons provided above, it is respectfully submitted that 
the rejections set forth in the Final Office Action of October 13, 2003, in 
connection with the subject application should be reversed. 

Favorable consideration of this Brief is respectfully requested. 



Respectfully submitted, 




D6vid S. Lee 
Registration No. 38,222 



LEE & HAYES, PLLC 
816 Second Avenue 
Seattle, WA 98104 
(206)315-4001 ext. 102 
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APPENDIX OF CLAIMS ON APPEAL 



1. One or more computer-readable media . having stored thereon a 
pluraUty of instructions that, when executed by one or more processors of a 
computer, causes the one or more processors to perform the following acts: 

receiving search criteria; 

generating a query vector based on text features of the search criteria; 

identifying media content pieces to be rendered by comparing the query 
vector to text feature vectors associated with a plurality of media content pieces; 

receiving user feedback regarding the relevancy of the identified media 
content pieces; 

modifying the query vector based on the user feedback; 

modifying one or more of the text feature vectors associated with the 
plurality of media content pieces based on the user feedback; and 

identifying new media content pieces to be rendered by comparing the 
modified query vector to the text feature vectors, including the one or more 
modified text feature vectors, associated with the plurality of media content 
pieces. 

2. One or more computer readable media as recited in claim 1, further 
comprising: 

generating another query vector based on one or more low-level features 
of the search criteria; and 

wherein the identifying comprises. 
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comparing the query vector to text feature vectors associated with the 
pluraUty of media content pieces to generate first results, 

comparing the other query vector to other low-level feature vectors 
associated with the plurality of media content pieces to generate second results, 
and 

combining, for one of the plurality of media content pieces, the first and 
second results corresponding to the one media content piece. 

3. One or more computer readable media as recited in claim 2, further 
comprising altering, based on the user feedback, a weighting of the results used 
in the combining. 

4. One or more computer readable media as recited in claim 3, 
wherein the altering comprises: 

determining, for the one of the plurality of media content pieces, whether 
the first result corresponding to the one media content piece is greater than the 
second result corresponding to the one media content piece; and 

weighting the first result corresponding to the one media content piece 
more heavily if the first result corresponding to the one media content piece is 
less than the second resuh corresponding to the one media content piece, and 
otherwise weighting the second result corresponding to the one media content 
piece more heavily. 
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5. One or more computer readable media as recited in claim 1, 
wherein modifying one or more of the text feature vectors associated with the 
plurality of media content pieces based on the user feedback comprises altering a 
weighting of one or more elements in the feature vector based on the user 
feedback. 

6. One or more computer readable media as recited in claim 1, 
wherein the search criteria comprises one or more words. 

7. One or more computer readable media as recited in claim 1, 
wherein the piece of media content comprises an image. 

8. One or more computer readable media as recited in claim 1, 
wherein the piece of media content comprises a portion of audio content. 

9. One or more computer readable media as recited in claim 1, 
wherein the piece of media content comprises a portion of multimedia content. 

10. A method comprising: 
identifying a media content source; 

collecting one or more pieces of media content and associated text from 
the media content source; 

extracting, for a piece of media content, one or more text features from the 
associated text; and 
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making the one or more text features available for searching. 

11. A method as recited in claim 10, further comprising: 
generating one or more text feature vectors from the extracted one or more 

text features; and 

wherein the making comprises making the one or more text feature 
vectors available for searching. 

12. A method as recited in claim 10, further comprising: 

extracting one or more low-level features from the media content piece; 

and 

making the one or more low-level features available for searching. 

13. A method as recited in claim 10, further comprising, for each 
collected piece of media content: 

classifying the image as meaningful or not meaningful; and 

wherein the extracting comprises extracting the one or more text features 

for the piece of media content only if the piece of media content is classified as 

meaningful. 

14. A method as recited in claim 10, wherein the media content source 
comprises a web site including a plurality of web pages, each web page including 
a plurality of pieces of media content and text associated with one or more of the 
plurality of pieces of media content. 
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15. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a filename and the one or more text features 
comprises one or more words in the filename. 

16. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a uniform resource locator (URL) and the one 
or more text features comprises one or more words in the URL. 

17. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises alternate text that can be displayed in place of 
the media content, and the one or more text features comprises one or more 
words of the alternate text. 

18. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises text surrounding the piece of media content on 
a web page, and the one or more text features comprises one or more words of 
the text surrounding the piece of media content. 

19. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a title of a web page that includes the piece of 
media content, and the one or more text features comprises one or more words in 
the title. 
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20. A method as recited in claim 10, wherein the associated. text for a 
piece of media content comprises a link on a web page that includes the piece of 
media content, and the one or more text features comprises one or more words in 
the link. 

21. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises anchor text corresponding to the piece of 
media content, and the one or more text features comprises one or more words in 
the anchor text. 

22. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises an image annotation corresponding to the piece 
of media content, and the one or more text features comprises one or more words 
in the image annotation. 

23. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises an image. 

24. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises a piece of audio content. 

25. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises a piece of multimedia content. 
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26. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
10. 

27. A method comprising: 
receiving search criteria; 

generating a query vector based on the search criteria; 

comparing the query vector to a feature vector corresponding to a piece of 
media content and having been generated based on text associated with the piece 
of media content; and 

determining, based at least in part on a result of the comparing, whether to 
render the piece of media content to a user. 

28. A method as recited in claim 27, further comprising: 
rendering a plurality of pieces of media content to the user; 

receiving feedback from the user regarding one or more of the plurality of 
pieces of media content; 

modifying the query vector based on the feedback; and 

repeating the comparing and determining based on the modified query 

vector. 

29. A method as recited in claim 27, further comprising: 
generating another query vector based on one or more low-level features 

of the search criteria; 
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comparing the other query vector to another feature vector corresponding 
to the piece of media content and having been generated based on the one or 
more low-level features of the piece of media content; 

combining a result of the other query vector to the other feature vector 
with the result of comparing the query vector to the feature vector; and 

wherein the determining comprises determining, based at least in part on 
the combined result, whether to render the piece of media content to the user. 

30. A method as recited in claim 29, further comprising: 

accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

altering, based on the user feedback, a weighting of the results during the 
combining. 

31. A method as recited in claim 30, wherein the altering comprises: 
determining whether a first distance between the other query vector and 

the other feature vector is greater than a second distance between the query 
vector and the feature vector; and 

weighting the result of the comparing the other query vector to the other 
feature vector more heavily if the first distance is less than the second distance, 
and otherwise weighting the result of the comparing the query vector to the 
feature vector more heavily. 

32. A method as recited in claim 27, further comprising: 
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accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

modifying the feature vector corresponding to the piece of media content 
based on the user feedback. 

33. A method as recited in claim 32, further comprising: 
generating a user space vector corresponding to each of the one or more 

pieces of meclia content for which user feedback is accessed; and 

using the user space vector corresponding to a particular piece of media 
content to modify the feature vector corresponding to the particular piece of 
media content. 

34. A method as recited in claim 27, further comprising: 

accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

altering a weighting of one or more elements in the feature vector based 
on the user feedback. 

35. A method as recited in claim 27, wherein the search criteria 
comprises one or more words. 

36. A method as recited in claim 27, wherein the piece of media 
content comprises an image. 
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37. A method as recited in claim 27, wherein the piece of media 
content comprises a piece of audio content. 

38. A method as recited in claim 27, wherein the piece of media 
content comprises a piece of multimedia content. 

39. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
27. 

40. A system comprising: 

a crawler module coupled to access a media content source and collect a 
plurality of media content pieces and associated text from the media content 
source; 

a feature extraction module coupled to extract one or more text features 
from one of the media content pieces; and 

a media content indexing module coupled to generate a text feature vector, 
based on the extracted one or more text features, corresponding to the one media 
content piece. 

41. A system comprising: 

a query generator to generate a query vector based on received search 
criteria; and 

a matching module coupled to, 



lee®hayes poc so9*324-9256 



34 



receive the query vector and compare the query vector to a plurality of 
feature vectors corresponding to a plurality of pieces of media content, wherein 
each of the plurality of feature vectors has been generated based on text 
associated with one of the plurality of pieces of media content, and 

identify one or more of the plurality of pieces of media content to return 
for rendering to a user based on the comparison of the query vector to the 
plurality of feature vectors. 

42. A method comprising: 
receiving search criteria; 

identifying, based at least in part on the search criteria, a piece of media 
content to be rendered; 

receiving user feedback regarding the relevancy of the rendered piece of 
media content; 

weighting for another piece of media content, based on the user feedback, 
both a result of comparing the high-level query vector to a high-level feature 
vector of the other piece of media content and a result of comparing the low-level 
query vector to a low- level feature vector of the other piece of media content; 
and 

combining the weighted result to determine whether to identify the other 
piece of media content for rendering. 
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43. A method as recited in claim 42, further comprising generating a 
new high-level query vector and a new low-level query vector based at least in 
part on the search criteria. 

44. A method as recited in claim 42, further comprising: 
generating a user space vector corresponding to the piece of media 

content; and 

using the user space vector corresponding to the piece of media content to 
modify the high-level feature vector corresponding to the piece of media content. 

45. A method as recited in claim 42, further comprising altering a 
weighting of one or more elements in the feature vector based on the user 
feedback. 

46. A method as recited in claim 42, wherein the high-level feature 
vector of the other piece of media content is a text feature vector. 

47. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
42. 

48. A method comprising: 

receiving user feedback regarding the relevancy of one or more pieces of 
rendered media content; and 
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for each of the one or more pieces of media content, 

modifying, based on the user feedback, a feature vector corresponding to 
the piece of media content, and 

making the modified feature vector available for subsequent searching of 
the one or more pieces of media content. 

49. A method as recited in claim 48, wherein the feature vector is a text 
feature vector. 

50. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
48. 

51. One or more computer-readable media having stored thereon a 
plurality of instructions that, when executed by one or more processors of a 
computer, causes the one or more processors to perform acts including: 

identifying a piece of media content to render to a user based at least in 
part on comparing a query vector corresponding to search criteria of the user and 
a feature vector corresponding to the piece of media content; 

receiving user feedback regarding the relevancy of the piece of media 
content; 

modifying the query vector based on the received user feedback; and 
modifying the feature vector based on the received user feedback in an 
off-line log mining process. 
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52. One or more computer-readable media as recited in claim 51 

wherein modifying the query vector comprises generating a vector ^ based on 

pieces of media content identified as relevant in the user feedback, and 

generating a new query vector ^"^^ according to the following: 
D^^=ilU^{\-il)D 

where ^ represents a confidence in the vector ^ . 

53. One or more computer-readable media as recited in claim 52, 
wherein modifying the query vector comprises generating a vector ^ based on 
pieces of media content identified as irrelevant in the user feedback, and 
generating a new query vector according to the following: 

54. One or more computer-readable media as recited in claim 51, 
wherein the piece of media content comprises one of: audio content, visual 
content, and multimedia content. 
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